With the advent of digital photography, consumers are amassing large collections of digital images and videos. The average number of images captures with digital cameras per photographer is still increasing each year. As a consequence, the organization and retrieval of images and videos is already a problem for the typical consumer. Currently, the length of time spanned by a typical consumer's digital image collection is only a few years. The organization and retrieval problem will continue to grow as the length of time spanned by the average digital image and video collection increases.
A user desires to find images and videos containing a particular person of interest. The user can perform a manual search to find images and videos containing the person of interest. However this is a slow, laborious process. Even though some commercial software (e.g. Adobe Album) permits users to tag images with labels indicating the people in the images so that searches can later be done, the initial labeling process is still very tedious and time consuming.
Face recognition software assumes the existence of a ground-truth labeled set of images (i.e. a set of images with corresponding person identities). Most consumer image collections do not have a similar set of ground truth. In addition, the labeling of faces in images is complex because many consumer images have multiple persons. So simply labeling an image with the identities of the people in the image does not indicate which person in the image is associated with which identity.
There exists many image processing packages that attempt to recognize people for security or other purposes. Some examples are the FaceVACS face recognition software from Cognitec Systems GmbH and the Facial Recognition SDKs from Imagis Technologies Inc. and Identix Inc. These packages are primarily intended for security-type applications where the person faces the camera under uniform illumination, frontal pose and neutral expression. These methods are not suited for use in personal consumer images due to the large variations in pose, illumination, expression and face size encountered in images in this domain.
In the article “Face Annotation for Family Photo Album Management”, International Journal of Image and Graphics, Vol. 3, No. 1 (2003), pp. 1-14 a face annotation with automatic recognition is described. However, this system does not solve the problem that arises when attempting to search for images containing more than one individual of interest. Specifically, this system does not solve the problem of searching for a set of images such that each of a set of individuals or objects of interest is contained within the set of images.